This Page Is Inserted by IFW Operations 
and is not a part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of 
the original documents submitted by the applicant. 

Defects in the images may include (but are not limited to): 

• BLACK BORDERS 

• TEXT CUT OFF AT TOP, BOTTOM OR SIDES 

• FADED TEXT 

• ILLEGIBLE TEXT 

• SKEWED/SLANTED IMAGES 

• COLORED PHOTOS 

• BLACK OR VERY BLACK AND WHITE DARK PHOTOS 

• GRAY SCALE DOCUMENTS 



IMAGES ARE BEST AVAILABLE COPY. 



As rescanning documents will not correct images, 
please do not report the images to the 
Image Problem Mailbox. 
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Translation quantity (u, v) = (1, 2) 



52c1 53c 



i — t — I — rVT- 

VI I I i \ I 

* i — i — x. — i — : 
A i i i i 
i i i i 
i — -i — r — i — 
iiii 



52c 



ft — \ — i — r 
iiii 
-i — i — 4. — u 
iiii 
iiii 
r — i — -t — v 
i i i 



iiii fx I i i *v I 



i i i i i i i i 

I I J. I I JL I ± L 

I I I I I I I I I 
I I I I I I I I I 

f— — I f 1 1 +■ 1 + 1- 

I I I I I I I I I 

I — I— 4— i— I— *—!—+— h- 

I I I I I I I I I 
I L — I 1 — I 1 — ± — \ 1- 



— I 
t 

H 

I 

.-I 
l 
I 



X 



52d 



y 

A 




i I 

I X I l L 



— I X I I J 

^X 



Rotation quantity ( 6 ) 



1 1 



Fig.6 



Image quality setting data 



Faster «<- 



Higher image quality 





Image quality 
mode setting 


1 


2 


3 


4 


5 


Example 1 — ► 


Image quality 
improvement V 


1 


2 


3 


4 


5 



V: Coefficient of 



Total number of pixels of frame data 
Total number of pixels of generated image data 





Image quality 
mode setting 


1 


2 


3 


4 


5 


Example 2 — * 


Threshold value TH1 


0.8 


0.6 


0.4 


0.3 


0.2 



TH1: Threshold value for average distance to 
closest pixel among pixels in plurality of 
frames of data 





Image quality 
mode setting 


1 


2 


3 


4 


5 


Example 3 — * 


Threshold value TH2 


1.5 


2 


2.5 


3.5 


4.5 



TH2: Threshold value for average number of frames 
of data having pixel in predetermined range 





Image quality 
mode setting 


1 


2 


3 


4 


5 


Example 4 — * 


Threshold value TH3 


0.6 


0.3 


0.2 


0.1 


0.05 



Number of pixels wherein number of pixels 
in predetermined range = 0 

TH3: 

Total number of pixels of generated image data 



.8 



Reference frame data 



Target frame data 



Z1(ix,iy+1) 

A 

ZKix.iy) Z1(ix+1,iy) 



P 

Z2(ix , ,iy , ) 



-> x 



Gradient method 



z 

A 



Z2(ix\iy') 
Z1(ix,iy) 




Z2(ix',iy') 
Z1(ix,iy) 




ly iy+1 
Ay 



PF04J81 1 



Fig.9 



y 



J 






(x'V) 








d./C** (x ' y) 










i 

0 





Fig.10 



Reference frame data 



Target frame data 





-» x 



-> X 



Pattern matching method 
Search for pixel minimizing 
|Z2i - Z1i| 



Target frame data 



Before shift 




After shift 



Translation quantity per pixel unit = (Ax, Ay) 



PF04J81 1 



Fig.11 ( 



START 



I 



Receive video file input 



Receive frame image 
selection input 



I 



Receive input of 
resolutions W, H, 
image quality mode 



i 



Acquire image quality 
improvement V 



I 



Determine number n1 of frames of 
data for acquisition 



n1 



W-H 
w h 



Acquire determined number of 
frames of data 

i — 



Camera motion 
estimation process 

i 



Frame conversion 
process 



Synthesis process 



i 



Display still image 



File save process 



c 



END 



3 



S105 



S110 



S115 



S120 



S125 



S130 



- S135 
S140 
S145 
S150 

S155 



PF04J81 1 




PF04J81 1 




PF04J81 1 



Fig.14 



C 



Camera motion 
estimation process 



Set reference frame data 



Set target frame data 



I 



Initialize variables 



Set position of pixels of 
interest i, i' from both frames 
of data 

i 



Calculate Pt, k, M20, Mn, M02, 
m u , m v , m 5 , c u , c v , c$ 



Calculate d 



1 



Calculate translation 
quantity u, v 

i 



Calculate rotation 
quantity d 



I 



S205 
S210 
S215 



S220 



S225 




S230 



S235 



S240 



S245 



Save as translation quantity data, 
rotation quantity data 




S250 



S255 



PF04J81 1 



Fig. 15 



Frame conversion 
process 



Set target frame data for 
coordinate conversion 



Set location of pixel of 
interest \ 



S305 



S310 



Convert location of pixel of 
interest i' so as to translate 
by (-u, -v) 



S315 



Convert location of pixel of 
interest i' so as to rotate 
by - 6 



S320 




S325 



NO 



YES 



All target frame 
data completed? 



S330 



YES 



Fig. 16 



Frame 12 3 4 




-> t 



Reference frame Rarget frames 



Frame 1 



< > 



Estimate of camera motion 



Frame 2 



Frame 3 



Frame 4 



Frame 1 




Superposition 



Fig. 17 



Synthesis process 

(D 



Set location of pixel of 
interest i for generating 
image data 



S405 



Calculate distance between pixel of 

interest i and pixels neighboring 
the pixel of interest i among all the 
pixels of frames of data 



S410 



Select shortest pixel 
closest to pixel of interest i 



Acquire distance 
MinL(i, f) to shortest pixel 



Acquire frame data 
containing shortest pixel 



Interpolation process 



NO 



S415 



S420 



S425 



S430 




S435 



YES 



Fig. 18 




o 



Pixel of generated image 
Pixel of Frame 1 <I> 
Pixel of Frame 3 ^ 



Pixel of Frame 2 
Pixel of Frame 4 




# Pixel of generated ^ Pixel of Frame 4 
image 



PF04J81 1 




Receive input of resolutions W, H, 
image qu ality mode 

i 



S505 



Acquire threshold value TH1 
for average Minl_(i, f) 



S510 



Acquire frame data 
H = 



S515 



Acquire next frame 
of data 



Camera motion 
estimation process 

i 



Frame conversion 
process 

i 



Synthesis process 



I 



Evaluation 
value V1 



I MinL( i,f ) 
W-H 



-S520 

S525 
S530 
S535 
-S540 




Display still image 
I 



File save process 



c 



END 



S555 
S560 



3 



Fig.21 



PF04J81 1 



C 



START 
1 



1 



Receive input of resolutions W, H, 
image quality mode 

1 



Acquire threshold value TH2 for 
average number nf(i) of frames of data 
having pixel within predetermined range 

I ~ 



S605 



S610 



Acquire frame of data 



J± 



S615 



Acquire next frame 

of data 

1 



Camera motion 
estimation process 



Frame conversion 
process 

i 



Synthesis process 
(2) 

I 



Evaluation 


Inf(i) 


value V2 


W-H 





-S620 
^-S625 
S630 
S635 




Display still image 



I 



C 



File save process 

I 



S655 
S660 



END 



Fig. 22 



PF04J81 1 



Fig.23 



Synthesis process 
(2) 



Acquire threshold value Lth 
of shortest pixel distance 



Establish location of pixel 
of interest i for generating 
image data 



S705 



S710 



Calculate distance between pixel 
of interest i and pixels neighboring 
pixel of interest among all pixels of 
frame da ta 

I 



S715 



Select pixels for 
which distance is equal to or 
less than threshold value Lth 



S720 



Acquire nf(i) 



S725 





i 






Acquire frame data containing 
selected pixel 














Interpolation process on 
per-frame data basis 


^ S735 



S730 



I 



Average each tone data after 
interpolation process 




S740 



S745 



PF04J81 1 



Fig.24 




0 Pixel of generated image 

O Pixel of Frame 1 O Pixel of Frame 2 

^ Pixel of Frame 3 Pixel of Frame 4 



PF04J81 1 



C 



START 
1 

I 
I 

I 



Receive input of resolutions W f H, 
image quality mode 

1 



S805 



Acquire threshold value TH3 
for ratio of number of pixels in 
predetermined range =0 

i 



S810 



Acquire frame of data 
l« = 



S815 



Acquire next frame of data 
I 



Camera motion 
estimation process 

i ~ 



Frame conversion 
process 

i 



Synthesis process 
(3) 



Evaluation 
value V3 



nu 



W-H 



- S820 

S825 
S830 
S835 
^- S840 




Display still image 



File save process 



c 



zrz 

END 



S855 
S860 



Fig.25 



PF04J81 1 



C 



Synthesis process 
(3) 



I 



) 



nu 



0 



S905 



Acquire threshold value Lth 
of shortest pixel distance 



Establish location of pixel 
of interest i for generating 
image data 



I 



S910 



S915 



Calculate distance between pixel 
of interest i and pixels neighboring 
pixel of interest among all pixels of 
frame data 



I 



S920 



Select pixels for 
which distance is equal to or 
less than threshold value Lth 



S925 




S935 



Acquire frame data containing selected pixel 



S940 



I 



Interpolation process 
on per-frame data basis 



S945 



Average each tone data after 
interpolation process 




S950 



S955 



Fig.26 



PF04J81 1 



Fig.27 




0 Pixel of generated image 
O Pixel of Frame 1 <I> Pixel of Frame 2 

^ Pixel of Frame 3 Pixel of Frame 4 

Pixel of Frame 5 



